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DETAILED ACTION 
Priority 

1 . This application claims benefit under the Patent Cooperation Treaty as a national 
stage application of International Application PCT/GB04/04687, filed 8 November 2004. 
This application also claims foreign priority of application GB 0326375.3 filed in the 
United Kingdom on 12 November 2003. 

Specification 

2. The title of the invention is not descriptive. A new title is required that is clearly 
indicative of the invention to which the claims are directed. The following title is 
suggested: "Object tracking within video images based on blob object models using 
Mahalanobis distance and Kalman filter". 

3. The drawings are objected to as failing to comply with 37 CFR 1 .84(p)(5) 
because they include the following reference character(s) not mentioned in the 
description: All of the instances of the form "step x.xx" in the specification are instead 
labeled "S.x.xx" in the drawings. The reference characters must match fully. Corrected 
drawing sheets in compliance with 37 CFR 1 .121(d), or amendment to the specification 
to modify the reference character(s) in the description in compliance with 37 CFR 

1 .121 (b) are required in reply to the Office action to avoid abandonment of the 
application. Any amended replacement drawing sheet should include all of the figures 
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appearing on the immediate prior version of tine slieet, even if only one figure is being 
amended. Each drawing sheet submitted after the filing date of an application must be 
labeled in the top margin as either "Replacement Sheet" or "New Sheet" pursuant to 37 
CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be 
notified and informed of any required corrective action in the next Office action. The 
objection to the drawings will not be held in abeyance. 

Claim Rejections - 35 USC § 101 

4. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

5. Claims 1-1 1 are rejected under 35 U.S.C. 101 because they do not fall within one 
of the four statutory categories of invention. While the claims recite a series of steps or 
acts to be performed, a statutory "process" under 35 U.S.C. 101 must (1) be tied to 
another statutory category (such as a particular apparatus), or (2) transform underlying 
subject matter (such as an article or material) to a different state or thing. The instant 
claims neither transform underlying subject matter nor positively tie to another statutory 
category that accomplishes the claimed method steps, and therefore do not qualify as a 
statutory process. In order for a process to be "tied" to another statutory category, the 
structure of another statutory category should be positively recited in a step or steps 
significant to the basic inventive concept, and not just in association with statements of 
intended use or purpose, insignificant pre- or post-solution activity, or implicitly. 
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6. Claim 12 is rejected under 35 U.S.C. 101 because the claimed invention is 
directed to non-statutory subject matter as follows. Claim 12 defines "a computer 
program or a suite of computer programs" embodying functional descriptive material 
(i.e., a computer program or computer executable code). However, the claim does not 
define a "computer-readable medium or computer-readable memory" and is thus non- 
statutory for that reason (i.e., "When functional descriptive material is recorded on some 
computer-readable medium it becomes structurally and functionally interrelated to the 
medium and will be statutory in most cases since use of technology permits the function 
of the descriptive material to be realized" - Guidelines Annex IV). 

The scope of the presently claimed invention encompasses products that are not 
necessarily computer readable, and thus not able to impart any functionality of the 
recited program. The examiner suggests amending the claim to embody the program on 
"computer-readable medium" or equivalent. Any amendment to the claim should be 
commensurate with its corresponding disclosure. 

Claim Rejections - 35 USC § 102 

7. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

8. Claims 1-4 and 12-17 are rejected under 35 U.S.C. 102(b) as being anticipated 
by Park, et al., "Segmentation and Tracking of Interacting Human Body Parts under 
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Occlusion and Sliadowing", Proc. Worl^sliop on Motion and Video Computing, pp. 105- 
1 1 1 (published 5 December 2002, hereinafter Park). 

9. With respect to claim 1 , Park discloses a method for tracking objects in a 
sequence of video images, comprising the steps of: 

storing object models relating to objects detected in previous video images of the 
sequence, the object models comprising values of characteristic features of the 
detected objects and variances of those values (implicit in the formation of blob-based 
object models and subsequent processing of those models for object tracking, section 
III); 

receiving a further video image of the sequence to be processed (implicit in the 
tracking of blobs throughout a video image, section III); 

detecting objects in the received video image ("A Gaussian mixture model is 
used to classify individual pixels.... Markov Random Field framework is used at the blob 
level to merge the pixels.... The blobs are then grouped to form the meaningful body 
parts by a simple body model," section I); 

determining characteristic features of the detected objects (unary, binary, and 
tertiary features listed in section III.B); 

calculating a distance measure between each detected object and each object 
model on the basis of the respective characteristic features using a distance function 
which takes into account at least the variance of the characteristic features (Eq. 9, 10, 
11); 
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matching tine detected objects to tine object models on tine basis of tlie calculated 
distance measures (Fig. 3); and 

updating the object models using the characteristic features of the respective 
detected objects matched thereto (Fig. 5). 

1 0. With respect to claim 2, Park discloses the method according to claim 1 , wherein 
the distance measure is a scaled Euclidean distance (Eq. 9, 10, 11). 

1 1 . With respect to claim 3, Park discloses the method according to claim 2, wherein 
the distance function is of the form: 



for object model I and detected object k, where xn and yki are values of the characteristic 
features of a stored object model and a detected object respectively, is the 
corresponding component of the variance of each feature, and the index i runs through 
N features of an object model (mathematically equivalent to Eq. 11). 

12. With respect to claim 4, Park discloses the method according to claim 1 , wherein 
the distance measure is the Mahalanobis distance (Eq. 11). 




13. Claims 12 and 13 are rejected for the same reasons set forth in the rejection of 
the corresponding method in claim 1. 
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14. Claims 14-17 are rejected for tine same reasons set fortli in tine rejection of tine 
corresponding method in claims 1-4, respectively. 

Claim Rejections - 35 USC § 103 

1 5. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary sl<ill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

16. Claims 5 and 18 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Park as applied to claims 1 and 14 above, respectively, and further in view of 
Marcenaro et al., "Multiple Object Tracking under Heavy Occlusions by Using Kalman 
Filters Based on Shape Matching," Proc. 2002 Int'l. Conf. on Image Processing, Vol. 2 
pp. 341-344 (published 22 September 2002, hereinafter Marcenaro). 

1 7. With respect to claim 5, Park discloses the limitations of claim 1 , but does not 
disclose predicting the values of charactehstic features of object models for the received 
frame. 

However, Marcenaro discloses using a Kalman filter, which predicts the values of 
characteristic features and then uses those feature predictions to update the object 
models (section 4). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to combine the technique of Park with the application of 
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the Kalman filter as disclosed by Marcenaro, motivated by the reduction in sensitivity to 
noise that the Kalman filter provides, a property of Kalman filters that is well known in 
the art. 

18. Claim 18 is rejected for the same reasons set forth in the corresponding method 

in claim 5. 

19. Claims 6 and 19 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Park as applied to claims 1 and 14 above, respectively, and further in view of Harville, 
U.S. Patent 7,003,136 B1 (filed as application 10/133,151 on 26 April 2002, hereinafter 
Harville). 

20. With respect to claim 6, Park discloses the limitations of claim 1 , but does not 
disclose increasing the variances of characteristic feature values when an object model 
Is not matched to a detected object. 

However, Harville discloses incrementing the state variances when no detected 
object is matched to an object model (col. 14, lines 17-22). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to combine the technique of Park with the increase in state 
variances disclosed by Harville, motivated by the need to decrease the level of tracking 
confidence for the person (Harville, col. 14 lines 20-21) and to accommodate a larger 
valid area for reacquiring the tracked object. 
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21 . Claim 19 is rejected for the same reasons set forth in the corresponding method 
in claim 6. 

22. Claims 7 and 20 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Park as applied to claim 1 and 14 above, respectively, and further in view of Magarey, 
U.S. Patent 7,177,446 B2 (filed as application 10/207,140 on 30 July 2002, hereinafter 
Magarey (Patent)). 

23. With respect to claim 7, Park discloses the limitations of claim 1 , but does not 
disclose updating the characteristic feature values with the average values found over a 
predetermined number of previous images whenever the object model is not matched 
with a detected object. 

However, Magarey (Patent) discloses updating reference data with the average 
of feature vectors of prior frames when determining that the track of the feature has 
been lost (col. 16 lines 1-5). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to combine the technique of Park with the average value 
update of Magarey (Patent), motivated by the need to update the tracked object with a 
feature vector representative of its past behavior in order increase the likelihood of 
reacquiring tracking at a later time. 

24. Claim 20 is rejected for the same reasons set forth in the corresponding method 
in claim 7. 
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25. Claims 8 and 21 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Park as applied to claim 1 and 14 above, respectively, and further in view of Senior et 
al., "Appearance Models for Occlusion Handling", Proc. 2"^ IEEE Int'l. Workshop on 
Performance Valuation of Tracking and Surveillance, pp. 108-1 15 (published 9 
December 2001 , hereinafter Senior). 

26. With respect to claim 8, Park discloses the limitations of claim 1 , but does not 
disclose considering an object to be occluded if an overlap with another object is 
detected. 

However, Senior discloses the determination of occlusion when objects overlap 
(Fig. 4, 5, and section 6), and further describes the resolution of such occlusion 
determinations. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to combine the technique of Park with the occlusion 
resolution of Senior, motivated by the increased accuracy of tracking when the tracking 
process can distinguish at least partially from among overlapping objects, a necessity 
when dealing with real world scenarios (Senior, abstract and section 1). 

27. Claim 21 is rejected for the same reasons set forth in the corresponding method 
in claim 8. 
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28. Claims 9 and 22 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Park as applied to claim 1 and 14 above, respectively, and further in view of Lo, et al., 
U.S. Patent 5,109,435 (issued 28 April 1992, hereinafter Lo). 

29. With respect to claim 9, Park discloses the limitations of claim 1 , but does not 
disclose outputting a signal to indicate that an object has been tracked for a certain 
number of frames. 

However, Lo discloses setting an acquisition flag when an object has been 
tracked for at least three frames (col. 6 lines 55-57) which is then provided as an output 
(Fig. 3, output from element 26). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to combine the technique of Park with the acquisition 
signal of Lo, motivated by the desire to interface with external devices which can act on 
the presence of the acquisition signal (e.g., by providing line of sight rates, object image 
growth rates, etc.; Lo, col. 6 line 64 through col. 7 line 2). 

30. Claim 22 is rejected for the same reasons set forth in the corresponding method 
in claim 9. 

31 . Claims 10 and 23 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Park as applied to claim 1 and 14 above, respectively, and further in view of 
Magarey, Patent Application Publication 2003/0053661 (published 20 March 2003, 
hereinafter Magarey (PgPub)). 
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32. With respect to claim 1 0, Parl< discloses the limitations of claim 1 , but does not 
specifically disclose deleting the object model when the object has not been tracked for 
a number of consecutive frames. 

However, Magarey (PgPub) discloses ending a tracking process once the 
tracked object has not been detected for a predetermined number of consecutive 
frames, with the implication that the object model is subsequently deleted. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to combine the technique of Park with the tracking process 
termination of Magarey, motivated by the increased efficiency in terms of memory and 
processor time that arises as a result of not continuing to track objects which have been 
determined to have likely left the field of view, as well as the ability to continue tracking 
objects which may have only temporarily become occluded or otherwise undetectable. 

33. Claim 23 is rejected for the same reasons set forth in the corresponding method 
in claim 10. 

34. Claims 11 and 24 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Park as applied to claim 1 and 14 above, respectively, and further in view of Sittler, 
R.W., "An Optimal Data Association Problem in Surveillance Theory," IEEE 
Transactions on Military Electronics, Vol. 8 Issue 2, pp. 125-139 (published April 1964, 

hereinafter Sittler). 
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35. With respect to claim 1 1 , Parl< discloses the limitations of claim 1 , but does not 
disclose storing a new object model when a detected object does not match a current 
object model. 

However, Sittler discloses the initiation of tracks during the course of monitoring 
a series of images, creating a new object model when a new object is detected (section 
IV.B.2). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to combine the technique of Park with the additional target 
tracking technique of Sittler, motivated by the desire to track new objects that arrive in 
the field of view. 

36. Claim 24 is rejected for the same reasons set forth in the corresponding method 
in claim 11. 

Conclusion 

37. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

Chodos et al., U.S. Patent 4,849,906. 
Fiala, H.E., U.S. Patent 5,109,435. 

Haritaoglu, I., et al. (September 1999) "Hydra: Multiple people detection and 

tracking using silhouettes." Proc. 1999 Int'l. Conf. on Image Analysis and Processing, 
pp. 280-285. 
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Any inquiry concerning tliis communication or earlier communications from tine 
examiner should be directed to Barry Drennan whose telephone number is 571-270- 
7262. The examiner can normally be reached on Monday through Thursday and 
alternate Fridays from 8:30am to 4:00pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Abul Azad can be reached on 571-272-7599. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 

Patent Application Information Retrieval (PAIR) system. Status information for 

published applications may be obtained from either Private PAIR or Public PAIR. 

Status information for unpublished applications is available through Private PAIR only. 

For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 

you have questions on access to the Private PAIR system, contact the Electronic 

Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 

USPTO Customer Service Representative or access to the automated information 

system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Barry Drennan/ /ABUL AZAD/ 

Examiner, Art Unit 4133 Supervisory Patent Examiner, Art 

Unit 4133 



